An Encompassing View on Tangible Interaction: A Framework

Our current understanding of human interaction with hybrid or augmented environments is very limited. We here focus on ‘tangible interaction’, denoting systems relying on embodied interaction, tangible manipulation, physical representation of data, and embeddedness in real space. This synthesis of prior ‘tangible’ definitions allows us to address a larger design space integrating approaches from different disciplines. We introduce a framework that contributes to understanding the (social) user experience of tangible interaction. This understanding lays the ground for evolving knowledge on collaboration-sensitive design. INTRODUCTION Tangible User Interfaces (TUIs) and Tangible Interaction are terms increasingly gaining currency within HCI. Through embedding computing in the everyday environment and supporting intuitive use these approaches share goals with other novel approaches to HCI. Variations have been pursued over the last two decades as ‘graspable user interfaces’ [7], ‘tangible user interfaces’ [19], or ‘tangible interaction’ [3, 5]. Design in this domain requires not only designing the digital but also the physical, as well as designing new types of interaction: these are new challenges for design and HCI. Through various effectws these systems lend themselves to the support of face-to-face social interaction, reflected in a considerable number of systems aimed at cooperative scenarios [1, 18, and see 19]. Research until recently focused on developing new systems. A move towards concepts and theory can be detected from a special issue on ‘tangible interfaces in perspective’ [10]. However, attempts to develop frameworks have concentrated mainly on defining terms or on categorizing and characterizing systems (e.g. [17, 19]). While supporting structural analysis, mapping out the design space and detecting uncharted territory, these offer little advice when designing for real world situations and seldom address users’ interaction experience. There is still a need for conceptual frameworks, that unpack why ‘tangible interaction’ works so well for users [6]. Equally there is a need for principled approaches supporting research and design of these new hybrid environments. We have chosen to use ‘tangible interaction’ as an umbrella term, drawing together several fields of research and disciplinary communities. This view encompasses a broad scope of systems relying on embodied interaction, tangible manipulation and physical representation of data, being embedded in real space and digitally augmenting physical space. It covers approaches from HCI, computing, product design and interactive arts. The proliferation of computing into everyday appliances draws product designers towards IT product design [3, 5]. Artists and museums experiment with hybrid interactives. Increasingly systems are developed by users e.g. in architecture or biology. This becomes even more prominent with computing moving beyond the desktop and ‘intelligent’ devices spreading into all fields of life and work. Thus a conceptual understanding of this new interface type and knowledge supporting design becomes even more important. In this position paper we can only give a short overview of our framework that focuses on the user experience of interaction and aims to unpack the interweaving of the material/physical and the social aspects of interaction. It is described in more detail in [13]. The framework contributes to the larger research agenda of Embodied Interaction [6, 15], offering four themes and a set of concepts. It builds upon results from a PhD project [11] and recent studies. One theme is described in detail in [12]. A BROAD VIEW ON TANGIBLE INTERACTION We now give an overview of the dominant views and approaches on ‘tangible interaction’ and propose a deliberately broad, encompassing view. A look at the above mentioned approaches from other disciplines reveals that the ‘tangible interface’ definition frequently used in HCI is too narrow to encompass these. From the characterizations found, we can distinguish a data-centered view, pursued in Computer Science and HCI; an expressive-movementcentered view from Industrial and Product Design; and a space-centered view from Arts and Architecture: • Data-centered view: [6, 10, 19] define ‘tangible user interfaces’ as utilizing physical representation and manipulation of digital data, offering interactive couplings of physical artifacts with “computationally This position paper is based on the first pages of a full paper accepted for CHI 2006, being shortened and slightly altered. CHI 2006, April 22–28, 2006, Montréal, Québec, Canada. Copyright 2006 ACM 1-59593-178-3/06/0004...$5.00. mediated digital information” [10]. This characterization of TUIs is dominant in HCI publications. Conceptual research from HCI and computer science tends to explores types of coupling and representations [19]. • Expressive-Movement-centered view: An emerging ‘school’ in product design aims to go beyond form and appearance and to design the interaction itself. This view emphasizes bodily interaction with objects, exploiting the “sensory richness and action potential of physical objects”, so that “meaning is created in the interaction” [5]. Design takes account of embodied skills, focuses on expressive movement and ‘rich’ interaction with ‘strong specific’ products tailored to a domain [3, 14]. The design community prefers the term ‘tangible interaction’. • Space-centered view: Interactive arts and architecture increasingly talk about ‘interactive spaces’. These rely on combining real space and real objects with digital displays or sound installations [2, 16]. “Interactive systems, physically embedded within real spaces, offer opportunities for interacting with tangible devices”, and “trigger display of digital content or reactive behaviors” [4]. Full-body interaction and use of the body as interaction device or display are typical for this approach. Tangible interaction, as we understand it, encompasses a broad scope of systems, building upon and synthesizing these approaches from different disciplinary backgrounds. These share the characteristics of tangibility/ materiality, physical embodiment of data, embodied interaction and bodily movement as an essential part of interaction, and embeddedness in real space [2, 3, 4, 5, 6, 10, 19]. This concept of tangible interaction has a broader scope than Ullmer and Ishii’s [19] description of tangible interfaces: “giving physical form to digital information” and its subsequent physical control, which is often used to define TUIs. Tangible interaction includes tangible appliances or remote control of the real world [14]. This approach focuses on designing the interaction itself (instead of the interface) and exploiting the richness of bodily movement [3]. Interaction with ‘interactive spaces’ by walking on sensorized floors or moving in space [2, 16] further extends our perspective, the body itself being used as input ‘device’. Instead of using a restrictive definition, it seems more productive to address this larger design space and to interpret these attempts at conceptualization as emphasizing different facets of a related set of systems. RELATED WORK ON ‘TANGIBLE’ FRAMEWORKS Previous attempts to develop frameworks have focused mainly on defining terms, categorizing and characterizing systems, on types of coupling. Most frameworks take a structural approach, systematically mapping out an abstract design space, but seldom address the human interaction experience. The most notable push towards a theory of tangible interaction and an understanding of the interaction experience, was provided by Dourish [6]. He emphasizes how social action is embedded in settings, focusing on the social construction of meaning. Thus materiality itself, and its relation to the social has been less discussed. Support of social interaction and collaboration might be the most important, domain-independent feature of tangible interaction, but this issue has attracted little explicit attention. The pioneering work by [1, 18] of analyzing social use of TUIs and identifying social affordances found few followers. Even though many researchers agree that TUIs are especially suited for co-located collaboration, conceptual work often only briefly mentions visibility of actions and distributed loci of control as collaborative affordances. Evaluations often assess individual use, or give primarily anecdotal accounts of field use. The research community therefore lacks concepts for analyzing and understanding the social aspects of tangible interaction and design knowledge on how to design so as to support social interaction and collaboration. This has motivated the development of our framework, which focuses on the (social) interaction experience, addressing the broader design space of ‘tangible interaction’. OUR FRAMEWORK ON TANGIBLE INTERACTION The framework is structured around four themes (figure 1) that are not mutually exclusive, but interrelated, offering different perspectives on tangible interaction. A set of concepts elaborates each theme, providing more concrete handles for understanding their implications. Themes are:. • Tangible Manipulation • Spatial Interaction • Embodied Facilitation • Expressive Representation We now (briefly) present the four themes, explaining each theme’s relevance for tangible interaction and presenting the related concepts, characterized with a short question in colloquial language. A more detailed description of themes and concepts can be found in the authors’ CHI paper [13]. Theme: Tangible Manipulation (TM) Tangible Manipulation refers to the reliance on material representations with distinct tactile qualities that is typical for tangible interaction. Tangible Manipulation is bodily interaction with physical objects. These objects are coupled with computational resources [19] to control computation. The main concepts, colloquially phrased, are: Haptic Direct Manipulation: Can users grab, feel and move ‘the important elements’? Lightweight Interaction: Can users proceed in small, experimental steps? Is there rapid feedback during interacting? Isomorph Effects: How easy is it to understand the relation between actions and their effects? Does the system provide powerful representations that transform the problem? Theme: Spatial Interaction (SI) Spatial Interaction refers to the fact that tangible interaction is embedded in real space and interaction therefore